Fixes #185.

Problem

With apps.lifecycle.pause_enabled=true, an app that is exited (not paused) was never revived on access — it 502-looped forever. The wake-on-access path dispatched on the stored app status: PAUSED woke via compose unpause, everything else via the start path. When an app exited out-of-band (core-upgrade converge stop, crash, OOM) while the database still said PAUSED, compose unpause errored container is not paused, the revive task crashed, and the app never started (first hit in production on shard c0p3x5, diagnostic 8aac62f1).

Fix

One idempotent revive primitive, start_app, that decides from the real container state (get_app_container_state via docker compose ps + docker inspect), not the stored status:

• running → no-op (reconciles the db status if it disagrees)
• paused → unpause (falls back to up -d if the stack can't be unpaused)
• exited / created / missing → up -d

It acts only on a revivable app (STOPPED/RUNNING/DOWN/PAUSED) and logs a warning whenever the db status and the real container state disagree (per the issue's follow-up comment about defensiveness). Both wake-on-access and the always-on control tick now call start_app only.

The @throttle decorator was changed from a single global timestamp to per-argument, so throttling one app's revive can no longer silently drop another app's wake — which is what forced the old wake path to special-case PAUSED to dodge the global throttle.

Recommended reading order

1. shard_core/util/misc.py — throttle → per-argument
2. shard_core/service/app_tools.py — get_app_container_state, start_app, _do_unpause/_compose_up/_mark_running
3. shard_core/service/app_lifecycle.py — route wake + always-on through start_app
4. agents.md — note the revive primitive
5. tests: test_throttle.py, test_app_compose_pinning.py, test_app_lifecycle_pause.py, test_app_lifecycle.py

Review panel

Adversarial reviewer — 1 BLOCKING, addressed:

• start_app dropped the old status allow-list; the always-on control tick could revive an app mid-uninstall/reinstall or in ERROR and recreate orphaned containers. Fixed (commit 9f03ca6): restored an explicit _REVIVABLE_STATUS gate (STOPPED/RUNNING/DOWN/PAUSED) with the real-state dispatch inside it; added test_start_app_skips_non_revivable_status.
• Mixed paused+exited multi-container stack: unpause leaves the exited sibling down or errors, re-creating the 502-loop (advisory). Fixed: the paused branch now falls back to up -d if unpause errors, so it can't crash-loop; added test_start_app_falls_back_to_up_when_unpause_fails. (The mixed state itself is pathological — compose pause is atomic — so full multi-container reconciliation is left out per simplicity.)
• docker inspect race could propagate instead of degrading to "missing" (advisory). Fixed: both docker calls are now inside the try.
• Per-app throttle adds hot-path cost / one-shot sidecar churn (advisory). Acknowledged; the running-app path is a cheap ps+inspect no-op, and single-container apps (the production case) aren't affected.

Test adversary — no blocking, gaps addressed:

• get_app_container_state never exercised against real docker. Fixed: test_get_app_container_state_reads_real_created_container installs an app and asserts the detector classifies its created container as needing a start (verified locally).
• Mixed paused+exited detector case + _compose_up recovery paths untested. Fixed: added parametrized cases and test_start_app_recovers_stale_containers.

DevEx / readability — advisory only:

• Duplication between start_app's paused branch and docker_unpause_app. Fixed: extracted _do_unpause.
• Bare-string states, docstring narrating bug history. Fixed: ContainerState Literal, trimmed docstring.
• start_app drops the docker_ prefix. Kept deliberately — it's the higher-level orchestrator vs. the thin docker_* CLI wrappers (matches the issue's proposed name).

CI fix: revive vs. uninstall race (commit 233858d)

First CI run surfaced a real regression the local run had misattributed to the port-80 collision: test_uninstall_running_app failed with DID NOT RAISE NotFound. Root cause — the new start_app reconcile recreates a missing/exited container for an app whose db status is still revivable, and the detached wake task (ensure_app_is_running) / always-on control tick could run compose up after the uninstall worker removed the containers, leaving an orphan that survives uninstall. The _REVIVABLE_STATUS gate alone couldn't close it (status read, then container removed, then compose up = TOCTOU).

Fix: a per-app asyncio.Lock (mirrors backup.py's BACKUP_IN_PROGESS_LOCK) held by both start_app and _uninstall_app, so a revive blocks until teardown finishes then reads status=None and skips. Added test_start_app_is_serialized_by_the_per_app_op_lock (falsifiable: fails without the lock).

Verification

Full unit suite green locally (57 app-lifecycle/compose/throttle tests incl. the new lock test). The two port-80 integration tests (test_app_starts_and_stops, test_uninstall_running_app) can't run on the dev VM (host port 80 held by infra); verified via CI, which has port 80 free.

CI: vendored type resync (commit 74e608f)

Controller main added ConfigOverrideKey/ConfigOverrideRequest and config_overrides_desired/applied after the last shard_core merge, so types-drift went red on every open PR. This branch carries the mechanical just get-types resync (only data_model/backend/shard_model.py, +26/-1, DO-NOT-MODIFY vendored) so it merges drift-free. Unrelated to the pause-revive fix — split into its own PR if preferred.